Incorporating Inertia Into Multi- Agent Systems 
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We consider a model that demonstrates the crucial role of inertia and stickiness in multi-agent 
systems, based on the Minority Game (MG). The inertia of an agent is introduced into the game 
model by allowing agents to apply hypothesis testing when choosing their best strategies, thereby 
reducing their reactivity towards changes in the environment. We find by extensive numerical 
simulations that our game shows a remarkable improvement of global cooperation throughout the 
whole phase space. In other words, the maladaptation behavior due to over-reaction of agents 
is removed. These agents are also shown to be advantageous over the standard ones, which are 
sometimes too sensitive to attain a fair success rate. We also calculate analytically the minimum 
amount of inertia needed to achieve the above improvement. Our calculation is consistent with the 
numerical simulation results. Finally, we review some related works in the field that show similar 
behaviors and compare them to our work. 

PACS numbers; 89.65.Gh, 02.50.-r, 05.40.-a, 89.75.-k 



I. INTRODUCTION 

There is a growing interest in studying artificial agents- 
interacting models which are able to generate global be- 
haviors found in social, biological and economical sys- 
tems 1]. Examples such as matching games and 
ideal gas models of trading markets ^Sj show that this 
approach commonly used by physicists can be nicely ap- 
plied to problems lay outside the discipline. One exciting 
fact is that these artificial models, although contain sim- 
ple governing rules, can still generate non-trivial global 
cooperative behaviors 0,13 • In these self-organized com- 
plex systems, agents can reach equilibrium states through 
adaptation, a dynamical learning process initiated by the 
feedback mechanism present in these systems. 

People possesses inertia when making decisions and 
switching strategies in economical systems. Conceptu- 
ally, this inertia is similar to the one used by Newton to 
describe the body motions in the physical world. It refers 
to how reluctant a person is going to drop his/her current 
economics plan and look for another one, just like an ob- 
ject is reluctant to change its motion state. Inertia may 
originate from: (1) the cost needed to change strategies, 
(2) the low sensitivity towards a change in environment 
and (3) the loss-aversion behavior in human — peo- 
ple loves to fight back from loss Like different bod- 
ies may have different mass in classical physical systems, 
different people may carry different inertia in economi- 
cal markets. In this paper, we introduce a simple model 
to study the idea of inertia. This model gives striking 
improvement of cooperative behavior, such as removal 
of maladaptation and dynamically increase of diver- 
sity among agents, without any necessity to alter initial 
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conditions and payoff mechanism. Actually, studies of 
a few variants of MG also show improvement in cooper- 
ations ^ 21j .12,, _13] . We shall further discuss their 
results and compare with ours in Section lVll after finished 
reporting our model and results. 

Our model is a modification of a famous econophysical 
model known as MG, proposed by Challet and Zhang in 
1997 MG is a simple game model that captures 

the minority seeking behavior found in stock markets 
and resources competitions. (See Refs. j3 ^ for 
an overview of econophysics and MG.) In MG, TV agents 
struggle to choose between two options repetitively, ei- 
ther buy (0) or sell (1) in each turn. Those who have 
chosen the minority sides are winners at that turn and 
are awarded 1 dollar, otherwise they lose 1 dollar. The 
only information they received is the history of the game, 
which is a binary bit string composed of the minority 
choices of previous M turns. A strategy is a map from 
the set of all possible histories to the set of two options. 
If a strategy predicts the minority correctly, it is added 
1 virtual score point, otherwise it loses 1 virtual score 
point. Each agent is assigned 5* strategies once and for 
all at the beginning of the game in order to aid his/her 
decision. In standard MG, an agent makes decision based 
on his/her best current strategy at hand, namely, the one 
with the highest virtual score. 

Clearly, there are 2*^ possible histories and hence 2^" 
available strategies. However, out of the whole strategy 
space, only 2*^+^ of them are significantly different. The 
diversity of the population is measured by a, which is 
equal to 2^^^ /NS. The smaller the a, the more similar 
are the strategies hold by agents. Up to first order ap- 
proximation, the dynamics of MG is determined by this 
control parameter a. [TMIl9l l20l| 

The most sparkling macroscopic observable in MG is 
perhaps the variance of option attendance per agents 
CT^/iV. It represents the wastage of the system and flue- 
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tuation of resources allocation; the smaller /N ^ the 
more the whole population benefits. Researchers found 
that a"^ /N falls below the value that all agents make 
their choices randomly in a certain range of a. This 
indicates that agents are cooperating although they are 
independent and selfish. More importantly, there is a 
phase transition at the critical point Uc which divides 
the a"^ /N against a curve into the so-called symmetric 
phase (a < etc) and asymmetric phase [a > ac). 



II. OUR MODEL 

To incorporate inertia into MG, we introduce a 
new modification - Hypothesis Testing Minority Game 
(HMG). Hypothesis testing is a standard statistical tool 
to test whether an effect emerged from an independent 
variable appears by chance or luck. In the standard ver- 
sion of MG, the best strategy is defined as the strategy 
with the highest virtual score. In HMG, however, an 
agent k determines his/her own best strategies by testing 
the following null hypothesis Hq: The current strategy 
Sk^o performs better than the other strategy S^^i avail- 
able to agent k. Note that we have restricted ourselves 
to the simplest case S = 2, but the model can be easily 
extended to S" > 2 cases under the same formalism. This 
agent possesses an sustain level /fe > 1/2 on his/her cur- 
rent strategy 5^,0, which is the same as the confidence 
level on the validity of the null hypothesis we commonly 
use in hypothesis testing (that is, the acceptance area of 
a standard normal). This Ik defines how much he/she 
could sustain the under-performance of Skfl and thereby 
represents his/her inertia. 

The Hq of a particular agent k can be quantitatively 
written as 



Hq : > Xk, 

Ok 



where 



+ 00 



"/^dx = Ik. 



(1) 



(2) 



Here, ^kjiTk) is the virtual score of a particular strategy 
Sk,j at Tfc, where Tk is the number of time steps counted 
from his/her adoption of Sk,o for that individual agent. 
The dominator 6k represents the fluctuation of strate- 
gies' performance the agent perceived. An agent k would 
continue to stick on his/her current strategy Sk,o until 
^k,jiTk) descends outside his/her sustain level. Then 
he/she has to admit that Hq is not likely to be true, 
rejects it and shift to the other strategy. After a change 
of strategy, the virtual scores of both strategies are reset 
to and Tk is set back to 1. 

The higher the value of Ik, the milder his/her response 
and the more reluctant for him/her in shifting strategies. 
In this way, Ik plays the role of inertia of an agent in this 
game. Agents with Ik = 1/2 would be most similar to 



standard MG agent, they employ strategy with the high- 
est virtual score. However, there are still two differences: 
these HMG agents would still stick on current strategy 
in case of a tie in virtual scores, and the virtual scores 
will be reseted after shifts in strategies. 

We remark that randomness are involved in only three 
places in HMG, namely, (1) the initial assignment of 
strategies and inertia; (2) the choice of a new strategy 
in case of a tie in the virtual scores of the alternative 
strategies when a player has decided to drop the current 
one; as well as (3) the determination of the winning side 
in case of a tie. Thus, the dynamics of HMG is deter- 
ministic when played by an odd number of agents each 
carrying 2 strategies. 



III. PURE POPULATION WITH RANDOM 
WALK APPROXIMATION 

We have performed extensive numerical simulation on 
our model. With the presence of inertia among agents, 
every agent needs a longer time to make up his/her mind 
and the equilibration time in HMG is lengthened. We 
take the value of every 1,000 time steps and regard 
the system as having equilibrated if the percentage differ- 
ence of successive measurement is smaller than e — 10^^. 
Upon equilibration, we take our measurement by record- 
ing the dynamics of the next 25,000 time steps. Further- 
more, we repeat this data taking procedure 150 times, 
each with an independent set of initial conditions. 

In a population where everyone tries to cling on the mi- 
nority side as long as possible, agents may have different 
inertia Ik , and some may have no inertia at all (standard 
MG agents) . We first study the behavior of HMG when 
every agent has the same Ik- (We shall move on to study 
the mixed population case in later sections.) We begin 
our study by determining the value of (5^, a perception 
of agents on the fiuctuation of virtual score difference 
between two strategies. A naive guess would be assum- 
ing flk,j{Tk) performs random walk for all strategies j 
throughout the game, then Sk equals %/2t^. 

Fig. n shows a plot of the variance of attendance for 
a particular option cr^/iV against the control parameter 
a for different inertia /, with Sk set to \f2n\. There is a 
huge drop of /N when / is sufficiently large, especially 
in symmetric phase when a is small (see Fig.lSJ. Not just 
the maladaptation in symmetric phase is greatly reduced, 
but the cooperation between agents is also improved in 
the asymmetric phase for certain values of /. 

The reduction of system wastage in the asymmetric 
phase (a > ac) is believed to be resulted by increasing 
stickiness of agents on current strategies and elongating 
their observing time. This leads to an increase of frozen 
agents (see Fig.Ol and an more effective crowd- anticrowd 
cancellation, succeeding in better cooperation. pH 123 . 

m 

However, things become more complicated when a < 
ac- From now on, this article will focus on the striking 
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FIG. 1: The variance of attendance per agent /N against 
the system complexity a for I equals (a) 0.53, (b) 0.6 and (c) 
0.9, setting 5k equals ^2Tfe. Here, iV = 501 and 5 = 2. The 
dashed line represents the jN curve in standard MG. 




FIG. 3: The frozen probability (\> against a. for different 7 by 
setting &k equals \/2rfc . The dashed line represents the frozen 
probability of the standard agents in MG. Here, N = 501, 5* 
= 2 and M = 5. 




FIG. 2: The variance of attendance per agent jN against 
the J at a = 0.06, setting equals \pli\. Here, TV = 501, S 
= 2 and M = 5. 



improvement of cooperation in the symmetric phase. The 
removal of maladaptation in this region is directly related 
to the disappearance of periodic dynamics that normally 
present in the standard MG. The periodic dynamics is 
a result of oversampling of strategy space and common 
zero initial conditions among agents when a < etc, ac- 
counting for the high volatility in the symmetric phase. 
It is reflected in a prominent period 2^^"*"^ peak in the au- 
tocorrelation of the attendance time series of a particular 
option BEllMllllll. Fig.Hshows an evidence of this 
postulate: as shown from the autocorrelation function, 
periodic dynamics appears in the case / — 0.53 which 
has high jN in Figs. [T] and |21 while the low a'^jN 
cases / = 0.6 and J = 0.9 show no trace of this signal. 

What is the critical limit of / in order to remove the 
maladaptation? To answer this, we have to look closely 
into the periodic dynamics that governs the maladapta- 
tion in the symmetric phase. Earlier study stated that 
virtual scores of strategies are likely to reset to every 
2M-1-1 ]2yjjj]-,ep of steps through the periodic dynamics in 
the symmetric phase. Initially all strategies have score 
point, whenever a strategy wins a bet in a particular 
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FIG. 4: The autocorrelation of attendance Co against various 
interval A on cases (a) standard MG, (b) 7 = 0.53, (c) 7 = 
0.6 and (d) 7 = 0.9 averaged over 50 independent runs. Here, 
iV = 501, M = 5 and S = 2. 



/i, most agents would rush to /? which is 2 score points 
ahead its anti-correlated partner /3 in the next appear- 
ance of \i. It is likely that they would lose due to this 
overcrowding. In this manner, the virtual scores of all 
strategies are reset at this stage. This loop repeats with 
interval 2*^+^ and leads to the large fluctuation of option 
attendance in the symmetric phase. |26j| 

Therefore, the question becomes when this reset and 
oscillate mechanism will disappear. Actually, the peri- 
odic dynamics is destroyed when agents are no longer 
sensitive enough to immediately shift to a strategy stand- 
ing out after winning a bet. The criteria for this situation 
to occur is given by: 



(3) 



where Xk satisfies Eq. (0). If the value of / satisfies the 
inequality Q, agents would no longer be constrained by 
the periodic dynamics every 2^^~^^ steps. Then, a re- 
recognizing process will draw in. In the standard MG, all 
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identical strategies have same virtual scores throughout 
the game. However, in HMG agents would clear all vir- 
tual scores after changing strategies. This move is done in 
multifarious time steps for different agents, depending on 
the combination of strategies in their hands. Hence, the 
scores of identical strategies eventually diverges if they 
are hold by different agents, and these strategies may 
be employed again in multifarious time in the future. 
The net effect of this re-recognizing process is diversi- 
fying strategies in the population intrinsically. In this 
way, both the oversampling and overcrowding found in 
the symmetric phase are relaxed, lowering the volatility. 

For instan ce, when M = 5, the limit Xc equals 
-2/V2 • 25+1 ^ -0.177; that is, / ~ 0.57. This criteria 
is confirmed in Figs.|2and0] — all cases that show no pe- 
riodic dynamics satisfies Eq. ijjjl and have low variances. 
Note that for the cases where / does not exceed this 
limit, their correlation signals are much stronger than 
that of the standard MG (see Fig. It is because 

the dynamics of HMG is more deterministic than that of 
the standard MG as HMG agents will continue to stick 
on current strategy when facing a tie in strategy virtual 
scores, which happens during a reset. That means their 
actions repeat during this reset and the system path is 
more likely to repeat, resulting in stronger correlation. 
This is like removing the random dice in standard MG 
when facing a tie in virtual scores, a periodic signal as 
strong as this case is also obtained. 



IV. PURE POPULATION WITH RUNTIME St 

Actually, the movement of the virtual score difference 
between two strategies is not likely to perform random 
walk. Another possible way in perceiving Sk is to put it 
as the actual standard deviation of this difference in run- 
time, which represents a more realistic market scenario. 
That is, 



Sk = 



(4) 

The results are very similar to the previous case, which 
are shown in Figs. EHZI However, the critical value of / 
for the system to escape from the grip of periodic dynam- 
ics appears to be higher. Remind that the virtual score 
difference of two strategies performs random walk with 
following step sizes and probabilities p: 



n 



k,Q 



^k,l — 







with p = 1/4, 
with p = 1/4, 
with p = 1/2. 



(5) 



Meanwhile, the presence of periodic dynamics ensure a 
reset every 2^~^^ number of time steps. We can approx- 
imately calculate the average variance by considering all 
possible traveling paths, which equals 2*^^+1/12 (detail 
mathematics is shown in the Appendix). For instance, 
when M = 5, the critical value for the periodic dynamics 



to disappear is < -2/^/25+1712 = -0.866; that is, 
/ ~ 0.81. This value of / is consistent with the data pre- 
sented in Figs. El and Again, we believe that after the 
breaking of periodic dynamics, the re-recognizing process 
mentioned in the previous section comes in and diversifies 
the strategy space, resulting in a drop of fluctuation. 




FIG. 5: The variance of attendance per agent /N against 
the system complexity a for I equals (a) 0.55, (b) 0.6 and (c) 
0.9, with 5k given by Eq. Q. Here, N = 501 and 5 = 2. The 
dashed line represents the /N curve in standard MG. 
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FIG. 6: The variance of attendance per agent /N against 
/ at a = 0.06, setting 5k satisfying Eq. Here, A'' = 501, 
S = 2 and A/ = 5. 



V. MIXED POPULATION WITH STANDARD 
MG AGENTS 

It is already clear that a pure population of agents hav- 
ing inertia reduces system wastage. Now it is instructive 
to study whether these agents (sticky agents) is advan- 
tageous over standard MG agents (sensitive agents) in a 
mixed population. 

Fig. |S1 gives the success rates of both races against 7 
in the mixed population with / = 0.9, where 7 is the 
fraction of sticky agents in the population. Clearly, these 
sticky agents take advantages of the sensitive agents for 
whole range of 7, they successes in maintaining their suc- 
cess rates close to 0.5. The sensitive agents are believed 
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(a) (b) 0.1 




FIG. 7: The autocorrelation of attendance Co against various 

interval i on cases (a) standard MG, (b) / = 0.55, (c) 7 — 0.6 FIG. 9: The autocorrelation of attendance Co for different 7 

and (d) / = 0.9 averaged over 50 independent runs. Here, TV averaged over 50 independent runs. Here, A'' = 501, PI = 5, 

= 501, M = 5 and 5 = 2 and 5k given by Eq. @. 5 = 2 and / = 0.9. 
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VI. PREVIOUS STUDIES IN THE 
LITERATURE AND COMPARISON WITH OUR 
RESULTS 

The reduction of the volatility by modifying the rules 
or the initial conditions of the standard MG is not a new 
idea in the field, especially for the symmetric phase. A 
few previous studies have shown results quite similar to 
that of the HMG. Here, we would like to first give a 
short review of these works and to compare them with 
our study. 



FIG. 8: A plot of success rate of sticky agents Wh, sensitive 
agents Ws and the whole population W against the fraction of 
sticky agents 7 in the mixed race. There are total 501 agents, 
with M = 5, 5" = 2 and J = 0.9. 



to be tightened by the periodic dynamics, making them 
to keep on losing. On the other hand, sticky agents are 
likely to win more frequently as they are resistant to fol- 
low the oscillation. Note that the whole population also 
benefits from adding in more sticky agents (see the tri- 
angles in Fig. ISJ. When 7 is increased up to about 0.6, 
Ws starts to rise. It is because the crowd of sensitive 
agents is no longer large enough to override the net ac- 
tions made by sticky agents, and therefore there is no 
more periodic dynamics existing. Fig. |51 confirms our 
suspicion, the periodic dynamics disappear around 7 ~ 
0.6. We have also performed simulations on mixed pop- 
ulation of sensitive agents and sticky agents with other 
values of /. As expected, sticky agents are only advan- 
tageous with I exceeds the critical value that allow them 
to escape from periodic dynamics mentioned in the last 
section. Otherwise, all agents in the whole population 
would still suffers from overcrowding and no one will be 
benefited. 



A. Thermal Minority Game 

Cavagna et al. proposed the Thermal Minority Game 
(TMG) 0, which adds stochasticity into the standard 
MG. In TMG, an agent does not employ the strategy 
with highest virtual score strict a way, rather he/she 
would use a strategy with probability calculated accord- 
ing to its virtual score and a fixed inverse temperature T. 
In other words, the employment of strategies by agents 
become probabilistic with the degree of stochasticity de- 
pending on T. They found that for certain range of T, 
the volatility in the game is reduced in most range of 
the control parameter a. That is, in both symmetric 
phase and asymmetric phases, TMG succeeds in raising 
the degree of cooperation between agents by introducing 
noises into the decision process of strategy selection for 
individual agents. 

In search of the continuous time dynamics of TMG, 
Garrahan et al. confirm by numerical simulation that 
the dynamics in the symmetric phase of MG is sensitive 
to initial conditions. In particular, they reported that 
the volatility would drop far from the original value if 
random initial conditions to strategies (with 0(1) initial 
virtual scores for a population of 100 agents) are assigned 
at the beginning of the game. ^3 
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B. Nash equilibrium 

In searching the replica solution and the Nash equilib- 
rium for the symmetric phase of the standard MG, Chal- 
let et al. found that the Nash equilibrium is not unique 
and agents at these equilibria use pure strategies (that 
is, they either always choose 1 or always choose 0). 
In Nash equilibrium, agents perform much better than in 
the standard MG, the volatility is greatly suppressed in 
the symmetric phase. 

C. Consideration of agents' own market impact in 

evaluation of strategy 

Challet et al. try to let the agents consider their own 
impact on the market during the evaluations of all strate- 
gies available to them. That is, the virtual score of 
a strategy is proportional to the cumulated payoff the 
agent would have received had he or she always played 
the strategy. Although the difference between this evalu- 
ation of virtual score and the original one is believed to be 
small (~ 1/^/N), the volatility is found to be far lower 
than the original MG. This difference is not negligible 
because of finite size effect and the high degree of over- 
sampling of the strategy space when a < Uc- However, 
this setting is computational intensive and unrealistic, as 
people in real market usually can only obtain information 
on his /her own current wealth and unlikely to try out all 
strategies. |II| 

D. The analytical solutions of batch minority game 

and the on-line minority game 

In batch minority game, the virtual score of a par- 
ticular strategy is updated as discrete accumulated ef- 
fect of order N iterations in the standard MG model, 
whereas the MG having the original updating method 
can be viewed as a "online" minority game in the neural 
network sense. After adding in stochasticity, initial eval- 
uations and generalizing these game to continuous time 
limit, Coolen's group has extensively written out the ana- 
lytical solutions of these two versions of MG. They found 
that in symmetric phase their theory pointed at the exis- 
tence of a critical value for the initial strategy valuations 
above the swtem would revert to a state with vanishing 
volatility. [Hill 

E. Introduction of diversity 

Wong et al. pointed out in Q that the maladaptation 
observed in the symmetric phase in the standard MG is 
originated from the fact that initial virtual scores of all 
strategies are the same. They then studied the effect of 
introducing diversity R/N into the game, where R is the 
range of randomly assigned initial scores to strategies at 



the beginning of the game and N is the number of agents. 
They found that by increasing the diversity, the maladap- 
tive behaviour observed in the symmetric phase a < ac 
is reduced and hence the cooperation among agents is 
promoted. 



F. Comparison to our model 

From the above studies, we can conclude that the 
volatility would suppressed under following conditions: 
(I) randomly allocating initial strategy score over a criti- 
cal value, (2) adding in noise or stochasticity in choosing 
a strategy, (3) assigning pure strategy or (4) taking an 
agent's impact on market into account when evaluating 
all strategies. 

Firstly, we would like to stress that the main focus in 
this article is to provide a simple formalism to incorpo- 
rate inertia into a multi-agent system such as MG, as 
well as recording its influence to the dynamics of the 
game. In HMG, there is no prior preference in strategies 
for they have the same initial virtual score. Unlike the 
standard MG, soon after the commencement of HMG, 
the preference of a strategy is determined by both the 
virtual score differences between strategies at hand and 
inertia Ik of agent k. Through the presence of inertia, 
each agent will gradually develop their own preference 
in strategies through dynamical adaptation. In this re- 
spect, even though the presence of inertia may eventually 
lead to difference views of an identical strategy between 
agents, this is achieved by an adaptive process through 
the dynamics of the system but not by artificially assign- 
ing a spread of initial virtual scores. This is a marked dif- 
ference between HMG and the works of Wong et aZ. |a| , 
Garrahan et al. |0 as well as Coolen et al. 0, Il3j| . 
More importantly, the spreading of initial virtual scores 
of strategies would only leads to a drop of volatility in 
the symmetric phase, but not the asymmetric phase. In 
HMG, however, there is a global improvement in both 
phases for certain value of /. 

We believed the TMG presents results most similar to 
our game. In both case, the degree of cooperation are 
raised in most range of a. However, as mentioned pre- 
viously, TMG achieve this by adding stochasticity and 
noise into agents' choice of best strategies. Meanwhile, 
in HMG agents are deterministic when choosing their 
best strategies: they stick to their current strategy until 
it is outperform to certain threshold, this does not in- 
volve any stochasticity. In fact, the dynamics of HMG is 
deterministic when played by an odd number of agents 
each carrying 2 strategies. 

Lastly, we think that using pure strategies and taking 
agents' themselves into account when evaluating all their 
strategies are impractical and unrealistic situations. Our 
model provide a natural, realistic way to prompt coop- 
eration, meanwhile demonstrating the effect of stickiness 
when people moving around investment strategies. 
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VII. CONCLUSIONS 

We have successfully introduced the concept of inertia 
into the Minority Game, which shows a remarkable im- 
provement of cooperation among agents in most range of 
a, especially in the symmetric phase a < ac- We also 
compare our findings with a few variants of MG reported 
in the literature. We calculated the critical values of in- 
ertia needed to uplift the cooperation behaviors, which 
depends on how agents perceive the fluctuation of vir- 
tual score difference between strategies. This reduction 
of sensitivity among agents is found to be useful in re- 
moving maladaptation due to over-reaction. In contrast, 
if every action is smooth and all agents response to infor- 
mation in no time, they will suffer from a overcrowd loss 
easily. Meanwhile, agents carrying stickiness seems to 
perform much better than sensitive agents. Our findings 
suggest that inertia (or stickiness) is crucial and bene- 
ficial to a society. It is hoped that the role of inertia 
will be investigated in more detail based on our model 
HMG, such as the effect of giving a diversifying range 
of inertia to a population. It is also instructive to apply 
our method of modeling inertia to study inertia effect in 
other multi-agent systems. 
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APPENDIX 

In this appendix, we consider a simple random walk 
of a cumulative sum xt after time t. At each step, Xt 
increases (decreases) by 1 if moves upward (downward) 
with probability 1/2. We also impose a boundary condi- 
tion that the sum is equal to at both t = and t = T. 
A schematic diagram is shown in Fig. IIUI Under such 
constraint, we find that the variance of Xt averages over 
all possible paths = *^ . Using this formula, we 
can evaluate the average standard deviation of virtual 
score difference of an agent's strategies within a period 
2A/+i^ that is the 5k mentioned in Eq. Q. 

First, we need to know the pro bability of xt within k 
and k + dk, which is given by 



P{k < Xt < k + dk) 



e-^'^'/'dk. 



(6) 



Hence, the probability of the cumulative sum xt to be 
within k and k + dk at time t and xt to be within / and 



I + dl at time T can be expressed by 

P{k < Xt < k + dk and I < xt < I + dl) 




FIG. 10: A schematic sketch showing a typical random walk 
of particle travels for T time step. 



= P{k < Xt < k + dk) ■ P{1 - k < XT-t < I - k + dl) 

(7) 

where t < T. The equality follows from the fact that 
the discrete steps size is equal to 1. Using the Eqs. © 
and 10, the conditional probability 



P{k <xt<k + dk\l <XT <l + dl) 
P{k < Xt < k + dk and I < xt < I + dl) 
P{1 < XT < I + dl) 



. J -^e-^(^-^y/(--^)dkdl 



(8) 



(9) 



By the boundary condition, < xt < dl, then we have 
P{k <xt<k + dk\0 <XT < dl) 

(10) 



2T 



nt{T - t) 



Therefore, the variance averaged over all possible 
paths: 

/ + OC- 
k'^P{k <xt<k + dk\0 < xt < dl) (11) 
-OO 

t{T-t) 



AT 



(12) 



In order to calculate we should rescale cr^ because 
the virtual score difference of an agent's strategies can 
move two steps upward (+2), two steps downward (—2) or 
keep stationary (0). Hence, by approximating the travel 
time T consists infinity number of time steps: 



61 = 2.'-n-^dt=L. 

T Jo AT 12 



(13) 



where Sk is the perceived fluctuation mentioned in 
Eq. Q. 
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